Assessment of the severity of acute pancreatitis. the usefulness of ROC analysis in comparative studies of clinical and imaging prognostic indices.

Dear Sir: We read the original article regarding assessment of the predictive value of the CT severity index (CTSI) for the severity of acute pancreatitis by Gürleyic et al. with great interest [1]. The purpose of this study was to make a comparative assessment of the accuracy of the CTSI proposed by Baltazar et al. [2], the APACHE II score and serum CRP concentrations in predicting the severity of acute pancreatitis. The cutoff values of 3 for the CTSI and 7 for the APACHE II were used in the present study, based on the results of studies which did not have exactly the same discrimination endpoint [2, 3]. The discrimination endpoint in this study is defined as the ability to separate those patients who had mild pancreatitis from those who had severe pancreatitis, according to the classification criteria of the 1992 Atlanta International Symposium [4]. Receiver operating characteristic (ROC) curves could have been used to determine the most appropriate cutoff point for the selected discrimination endpoint which corresponds to the best possible trade-off between sensitivity and specificity which were estimated in the present sample. Moreover, the area under the ROC curve is a reliable measure of overall predictive discrimination and a previously described method for comparison of the areas under the ROC curves, derived from the same cases, could also have been used [5, 6]. The additional information provided by ROC curves in studies of prognostic indices of acute pancreatitis severity derives from the complete illustration of the relationship between s ensitivity and specificity for a certain discrimination endpoint (severe vs. mild pancreatitis in this case). This might have been useful because: a) the clinical impact of the two types of misclassification (failure to correctly identify a case of severe pancreatitis or failure to correctly identify a case of mild pancreatitis) is not the same, and b) it is necessary to realize new prospective comparative studies for assessing the clinical impact of promising imaging techniques, such as MRI or contrast-enhanced US in the near future [7, 8]. Moreover, it should be noted that even though the Atlanta classification system provides a reliable basis for experimental studies for the clinical management of acute pancreatitis, it is not considered to be a perfect system since intermediate forms of the disease do occur [9]. If an imperfect gold standard is used, the estimated accuracy of the tests may suffer (" imperfect gold standard …